System and method for autonomous joint detection-classification and tracking of acoustic signals of interest

ABSTRACT

Systems and methods are disclosed for autonomous joint detection-classification of acoustic sources of interest. Localization and tracking from unmanned marine vehicles are also described. Based on receiving acoustic signals originating above or below the surface, a processor can process the acoustic signals to determine the target of interest associated with the acoustic signal. The methods and systems autonomously and jointly detect and classify a target of interest. A target track can be generated corresponding to the locations of the detected target of interest. A classifier can be used representing spectral characteristics of a target of interest.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application is a continuation of U.S. patent applicationSer. No. 15/495,536, filed on Apr. 24, 2017, which claims the benefit ofpriority under 35 U.S.C. § 119 from U.S. Provisional Patent ApplicationSer. No. 62/327,337 entitled “Autonomous, Embedded, Real-Time DigitalSignal Processing Method and Apparatus for Passive Acoustic Detection,Classification, Localization, and Tracking from Unmanned UnderseaVehicles (UUV) and Unmanned Surface Vehicles (USV)”, filed on Apr. 25,2016, the disclosures of which are hereby incorporated by reference intheir entirety for all purposes.

BACKGROUND

In underwater environments, acoustic signals are generated by a varietyof acoustic sources, with examples including, but not limited to, marinemammals, fish, meteorological or geological phenomenon, as well asmarine vehicles, such as military, recreational or commercial vehiclesoperating at or below the surface. Acoustic signals corresponding to aparticular source can be particularly difficult to detect due topresence of noise in the marine environment. For example, acoustic datameasured/recorded by a hydrophone can include significant noise (N) inaddition to signals (S) associated with an acoustic source of interest,such that the data is actually represent a sum of the signal(s) and thenoise (S+N). The noise components can be very significant, potentiallymasking many signals of interest. This problem of noise is oftenexacerbated by the typically anisotropic and temporally variablecharacter of noise in undersea environments. See, e.g., Wenz, G.,“Ambient Noise in the Ocean: Spectra and Sources,” J. Acoustic Soc. Am.,Vol. 34, no. 12 (December 1962), the entire content of which isincorporated herein by reference.

Equipment for analyzing or characterizing acoustic signals in marineenvironments, e.g., in sub-surface conditions, have typically eitherbeen tethered or linked via underwater acoustic communications (ACOMMS)to an above-surface antenna, or have been included in under-watervessels that have large computing resources and power reserves, e.g.,submarines. Generally, acoustic signal and target data in unmannedmarine environments must be recorded, stored, transmitted andpost-processed using a variety of complex disparate system components,such as sensors, transceivers, and computing devices before accuratelydetecting, classifying and tracking targets of interest. The ability toautonomously record, process and transmit marine acoustic signalsassociated with a target of interest in real-time has been limited bythe processing power, battery life, available memory, communicationcapabilities, and/or the signal processing algorithms associated with agiven vehicle platform or system implementation.

SUMMARY

Aspects and implementations of the subject technology of the presentdisclosure provide methods and systems for autonomously performingsimultaneous (or, “joint”) detection and classification of acoustictargets of interest; in some embodiments, acoustic targets of interestcan be tracked after the joint detection-classification occurs. Thejoint detection and classification (“detection-classification”) isperformed on acoustic data received from or provided by one or moreacoustic sensors, e.g., hydrophones. The collected acoustic datarepresent acoustic signals (and noise) received from targets, objectsand sources of interest present in a marine environment. Autonomouslyprocessing the acoustic signals in real-time from a marine vehicleplatform is accomplished utilizing a joint detection-classificationmethod according to an implementation of one aspect of the presentdisclosure.

In one aspect, the present disclosure relates to a method for autonomousjoint detection-classification, and tracking of targets of interest. Inone example, the method includes receiving acoustic signals from two ormore hydrophone sensors. The acoustic signals are characterized bytime-series data. The method includes detecting a target of interestfrom the received acoustic signals. The method further includestransforming the received acoustic signals from the time domain to thefrequency domain. The method also includes generating a relative-bearingbeam response in one or more steering directions. The method includesestimating the median background noise level of the relative-bearingbeam response and normalizing the spectral response by the estimatedbackground noise level. The method further includes performing a jointdetection-classification operation. The joint detection-classificationoperation is a selective frequency integration of the normalizedspectral responses associated with each relative-bearing beam response.The frequency integration is informed by the detailed physics of theunderlying classifier for the target of interest to satisfy a spatialand spectral hypothesis of the classifier. The method includes computinga detection surface in bearing and time based on the frequencyintegration of the joint detection-classification operation. The methodalso includes determining a decision surface in bearing and time bycalculating a constant-false-alarm-rate (CFAR) detection threshold fromthe estimated noise background level and applying theconstant-false-alarm-rate detection threshold to the detection surface.The method further includes generating a target track corresponding tothe decision surface. The method includes associating decision surfacethreshold exceedances to produce a relative-bearing track as a functionof bearing and time. The method also includes calculating a true bearingtrack corresponding to each of the relative-bearing tracks byreconciling the relative-bearing tracks with an estimate of hydrophonesensor array orientation. The method further includes transforming theacoustic signals from the frequency domain to the time domain using therelative-bearing track to generate the target track. The method alsoincludes generating a summary report of spectral data associated witheach target track and outputting a compressed data report identifyingthe characteristics of the generated target track included in thegenerated summary report.

In a further aspect, the present disclosure relates to a system forautonomous joint detection-classification, and tracking of targets ofinterest. In one example, the system includes a hydrophone sensor arrayconfigured to receive and transmit acoustic signals originating above orbelow the surface. The system further includes, a marine vehicleplatform including a memory module, a communications module, a globalpositioning system receiver and one or more embedded processorsconfigured to autonomously receive acoustic signals from hydrophonesensors. The acoustic signals are characterized by time-series data. Theembedded processors are further configured to detect a target ofinterest by transforming the acoustic signals from the time domain tothe frequency domain and generating a relative-bearing beam response foreach of the transformed signals. The embedded processors are alsoconfigured to estimate the median background noise level of therelative-bearing beam response and normalize the spectral response ofthe relative-bearing beam response by the estimated background noiselevel. The embedded processors are further configured to perform a jointdetection-classification operation. The joint detection-classificationoperation is a selective frequency integration of the normalizedspectral responses associated with each relative-bearing beam response.The frequency integration is constrained by the detailed physics of theunderlying classifier for the target of interest to satisfy a spatialand spectral target hypothesis of the classifier. The embeddedprocessors are configured to compute a detection surface in bearing andtime based on the frequency integration of the jointdetection-classification operation. The embedded processors are alsoconfigured to determine a decision surface in bearing and time bycalculating a constant-false-alarm-rate (CFAR) detection threshold fromthe estimated background noise level and applying theconstant-false-alarm-rate detection threshold to the detection surface.The embedded processors are configured to generate a target trackcorresponding to the decision surface by associating decision surfacethreshold exceedances to produce a relative-baring track as a functionof time. The embedded processors are also configured to calculate a truebearing track corresponding to each of the relative-bearing tracks withan estimate of hydrophone sensor array orientation. The embeddedprocessors are further configured to transform the acoustic signals fromthe frequency domain to the time domain using the relative-bearing trackand generate a summary report of spectral data associated with eachtarget track. The embedded processors are also configured to output acompressed data report identifying the characteristics of the generatedtarget track included in the generated summary report.

In another aspect, the present disclosure relates to a system forautonomous joint detection-classification, and tracking of targets ofinterest. In one example, the system includes a memory and a processorhaving access to the memory. The processor is configured to receiveacoustic signals from acoustic sensors. The acoustic signals arecharacterized by time-series data. The processor is further configuredto detect a target of interest by transforming the acoustic signals fromthe time domain to the frequency domain and generating arelative-bearing beam response for each of the transformed signals. Theprocessor is also configured to estimate the median background noiselevel of the relative-bearing beam response and normalize the spectralresponse of the relative-bearing beam response by the estimatedbackground noise level. The processor is further configured to perform ajoint detection-classification operation. The jointdetection-classification operation is a selective frequency integrationof the normalized spectral responses associated with eachrelative-bearing beam response. The frequency integration is constrainedby an underlying classifier for the target of interest to satisfy aspatial and spectral target hypothesis of the classifier. The processoris configured to compute a detection surface in bearing and time basedon the frequency integration of the joint detection-classificationoperation. The processor is also configured to determine a decisionsurface in bearing and time by calculating a constant-false-alarm-rate(CFAR) detection threshold from the estimated background noise level andapplying the constant-false-alarm-rate detection threshold to thedetection surface. The processor is configured to generate a targettrack corresponding to the decision surface indicating the track of thetarget of interest.

It is understood that other configurations of the subject technologywill become readily apparent to those skilled in the art from thefollowing detailed description, wherein various configurations of thesubject technology are shown and described by way of illustration. Aswill be realized, the subject technology is capable of other anddifferent configurations and its several details are capable ofmodification in various other respects, all without departing from thescope of the subject technology. Accordingly, the drawings and detaileddescription are to be regarded as illustrative in nature and not asrestrictive.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and related features, and advantages of the present disclosurewill be more fully understood by reference to the following detaileddescription, when taken in conjunction with the following figures,wherein:

FIG. 1 is a diagram of an exemplary system configuration for autonomousjoint detection-classification, and tracking of targets of interestusing a marine vehicle platform;

FIG. 2 is a flowchart of an exemplary method of autonomous jointdetection-classification, and tracking of targets of interest;

FIG. 3 is a flowchart of an exemplary method of autonomous jointdetection-classification of a target of interest corresponding to eachof the respective received acoustic signal;

FIG. 4 is a flowchart of an exemplary method of generating a targettrack corresponding to the decision surface;

FIG. 5 is a diagram of an exemplary system configuration for receivingacoustic signals using a marine vehicle platform;

FIGS. 6A and 6B are diagrams demonstrating the calculation of anacoustic signal's frequency-integrated beam response by conventional andjoint detection-classification methods as described herein.

FIG. 7 is a block diagram of a computing system in accordance with anillustrative implementation.

Like reference numbers and designations in the various drawings indicatelike elements.

In one or more implementations, not all of the depicted components ineach figure are required, and one or more implementations can includeadditional components not shown in a figure. Variations in thearrangement and type of the components can be made without departingfrom the scope of the subject disclosure. Additional components,different components, or fewer components can be utilized within thescope of the subject disclosure.

DETAILED DESCRIPTION

The detailed description set forth below is intended as a description ofvarious implementations and is not intended to represent the onlyimplementations in which the subject technology can be practiced. Asthose skilled in the art would realize, the described implementationscan be modified in various different ways, all without departing fromthe scope of the present disclosure. Accordingly, the drawings anddescription are to be regarded as illustrative in nature and notrestrictive.

Examples of systems and methods are described herein for autonomousjoint detection-classification and tracking acoustic sources of interestin real-time from collected acoustic data. Exemplary embodiments areshown and described in the context of marine environments where acousticsensor arrays (including two or more hydrophones) are deployed using amarine vehicle as a host platform. As described with particular focusherein, acoustic signal data can include data corresponding to thepropagation of sound through water or other liquids. In otherembodiments of the present disclosure, however, acoustic signal data caninclude data corresponding to the propagation of sound through air oranother medium.

According to embodiments of the present disclosure, one or more acousticsources (or targets) of interest are detected and classified jointly andthen tracked. The joint detection-classification and tracking can beperformed autonomously, without requiring any human intervention. Theseprocesses (functionality) can be realized and accomplished withlow-power and computationally efficient processing. Acoustic signals canbe processed to reduce or reject the noise that exists in a receivedsignal in order to facilitate joint detection-classification, andtracking of the desired signal sources or objects associated with signalof interest. For example, cod fish generate specific noises or acousticsignals during mating that can be detected and classified to assistscientists tracking the location or migration patterns of cod fishstocks in particular marine areas. In this example, scientists can usehydrophones to autonomously acquire sound data relating to cod fishmigratory patterns from undersea platforms and process the sound data todistinguish the sounds generated by mating cod fish from other acousticsounds or noise present in the marine environment (such as noise orsounds associated with other marine species which are not being studiedby scientists). A hydrophone is a microphone designed to record sound,or acoustic signals, underwater. The acoustic signals generated bytargets of interest and other noise sources in marine environments canbe received by a hydrophone or hydrophone sensor and digitized fortransmission to the marine vehicle platform for further processing. Amarine vehicle platform can be included in the system and can containembedded processors configured to receive and process acoustic datareceived from the hydrophone sensor. The embedded processors can includegenerally available computing components, such ascommercial-off-the-shelf hardware and software components configured foracoustic signal processing in marine environments. The embeddedprocessors can be configured with signal processing algorithms and logicto perform autonomous joint detection-classification, and tracking ofacoustic signals corresponding to particular targets of interest in areal-time manner.

In some implementations of the present disclosure, the jointdetection-classification functionality or the components executing thejoint detection-classification functionality operate jointly or inparallel to increase sensitivity to weak signals. For example, utilizinga joint detection-classification method makes use of all informationembedded in the measured acoustic signal thereby avoiding anyassumptions of spectral feature independence that can may be invoked inimplementations where detection, classification, and tracking areperformed sequentially. A consequence of sequential detection,classification, and tracking implementations is that the detectionfunctionality discards information related to the correlation orinterdependence of noise spectrum features radiated by the target ofinterest. While sequential implementations may reduce computationalcosts, key performance measures such as passive sonar recognitiondifferential may be limited, thus degrading overall system performance.

The embedded processors configured on the marine vehicle platform canperform autonomous joint detection-classification, and tracking of atarget of interest. The target of interest can generate an acousticsignal that is received by a hydrophone sensor configured in an array ofmultiple hydrophone sensors towed behind or mounted on the marinevehicle platform. The embedded processors on the marine vehicle platformare configured to receive the acoustic signal and apply a series ofsignal processing techniques to jointly or simultaneously detect andclassify the target of interest generating the acoustic signal as wellas track the location of the target of interest. In one implementation,a decision surface can be identified for the target of interestproducing the acoustic signal. Once the acoustic signal associated withthe target of interest has been jointly detected and classified, theembedded processors configured on the marine vehicle platform can trackthe location of the target of interest. A target track identifying theexact relative or time bearing to the target of interest can begenerated by associating detection surface threshold exceedances, ordetection events with similar spatial and spectral characteristics toproduce a relative-bearing track for the target of interest that hasbeen jointly detected and classified. The accuracy of therelative-bearing track can be improved by reconciling therelative-bearing track with an estimate of the orientation orpositioning of the hydrophone sensors. The embedded processors can befurther configured to generate a report summarizing the spectral dataassociated with each target track. The embedded processors can furtherbe configured to output or transmit the report and data describing thetargets of interest in formats that are suitable for exfiltration inbandwidth limited operating environments or communication channels.

In some implementations, the hydrophone sensors can vary in type,arrangement or orientation. For example, a plurality of hydrophonesensors can be configured in an array and can transmit acoustic signaldata to the marine vehicle platform on a telemetry cable. In otherimplementations, a hydrophone sensor array can be mounted to the marinevehicle platform. In some implementations, the hydrophone sensor arraycan be external to the marine vehicle platform. In otherimplementations, the hydrophone sensor array can be towed by the marinevehicle platform. Additionally, or alternatively, one or more hydrophonesensor arrays can be utilized and the received acoustic signals can beintegrated for autonomous processing in real-time using the system andmethod of joint detection-classification and tracking described herein.In some implementations, the marine vehicle platform can be propelledusing an electric-motor driven propeller. In other implementations, abuoyancy-driven propulsion engine can be utilized to propel the marinevehicle platform. In some implementations, the marine vehicle platformmay be propelled using wave energy. In other implementations the marinevehicle platform may be a vessel operating at the surface of the water.

FIG. 1 is a diagram of an exemplary system configuration for autonomousjoint detection-classification, and tracking of targets of interestusing a marine vehicle platform. In broad overview, the illustratedsystem configuration 100 includes (i) a marine vehicle platform 102,(ii) a signal processing system 104, and (iii) a hydrophone sensor array106. The hydrophone sensor array 106 includes individual hydrophonesensors 108 a-108 e (generally referred to as hydrophone sensors 108).The hydrophone sensors 108 are connected to marine vehicle platform 102by connection means 110.

Referring to FIG. 1, in more detail, the system configuration 100 forautonomous joint detection-classification, and tracking of targets ofinterest includes a marine vehicle platform 102 connected to hydrophonesensor array 106. Marine vehicle platform 102 can be an unmannedundersea vehicle (UUV) or an autonomous underwater vehicle (AUV). Marinevehicle platform 102 can be a submerged vehicle or a surfaced vehicle.The power supply for propelling the marine vehicle platform 102 can bean electrical power supply, such as an onboard battery. Additionally, oralternatively, the marine vehicle platform 102 can be propelled using abuoyancy-driven propulsion engine, such as a glider-based vehicle.Examples of marine vehicle platforms, such as marine vehicle platform102, include but are not limited to commercial off-the-shelf vehicleplatforms, e.g., the REMUS® 100 and REMUS® 600 vehicles manufactured byHydroid Inc., a Kongsberg Co. of Pocasset, Mass., the Bluefin UUVsmanufactured by Bluefin Robotics Corporation of Quincy, Mass., theSLOCUM G2, G3 and 200 glider vehicles manufactured by Teledyne WebbResearch of East Falmouth, Mass., the Seaglider™ vehicles manufacturedby the University of Washington Applied Physics Laboratory, and the WaveGlider® vehicles manufactured by Liquid Robotics of Sunnyvale, Calif.The signal processing components and functionality described herein canbe implemented on a variety of vehicle platforms and are not limited tospecific vehicle configurations.

As further shown in FIG. 1, the marine vehicle platform 102 includes asignal processing system 104. The signal processing system 104 includesa plurality of computing components configured to receive and processthe acoustic signals transmitted from the hydrophone sensor array 106.The plurality of computing components configured in signal processingsystem 104 include advanced signal processing algorithms capable ofautonomously receiving and processing acoustic signal data fromhydrophone sensor array 106 in real-time in order to jointly detect(e.g., detect sound sources) and classify (e.g., identify orcharacterize the acoustic signal source such as a marine mammal orgeophysical phenomenon, or a surface vessel) targets of interest. Theplurality of computing components configured in signal processing system104 are further configured to track (e.g., determine the location and/ortrajectories of targets of interest as functions of time), and output,or exfiltrate, target data for use elsewhere. The signal processingsystem 104 can receive power from the power supply included in themarine vehicle platform 102 or can include its own power supply.Additionally, the signal processing system 104 includes a digital datarecorder or memory capable of storing acoustic signal and target ofinterest data. The memory included in signal processing system 104 canalso store computer-readable instructions that when executed cause theplurality of computing components to perform autonomous jointdetection-classification, tracking and output of acoustic signal andtarget of interest data according to the method disclosed herein.

Still referring to FIG. 1, the system configuration for autonomous jointdetection-classification, and tracking of targets of interest 100includes the hydrophone sensor array 106. The hydrophone sensor array106 includes a plurality of individual hydrophone sensors, for examplehydrophone sensors 108 a-108 e, generally known as hydrophone sensors108. The hydrophone sensor array 106 can be towed by marine vehicleplatform 102 using low drag cables. In some implementations, thehydrophone sensor array 106 can be mounted to marine vehicle platform102. The means of connection can further include a data cable tofacilitate transmission of the digitized acoustic signal data collectedby hydrophone sensors 108 to the signal processing system 104. In someimplementations, the hydrophone sensor array 106 is a multi-channelhydrophone sensor array, where each hydrophone sensor 108 is configuredwith one or more communication channels to transmit acoustic signals tothe marine vehicle platform. In other implementations the hydrophonesensor array 106 is a linear or planar hydrophone sensor array. Theadvanced signal processing algorithms included in signal processingsystem 104 are capable of autonomously receiving and processing acousticsignal data according to the method described herein are furtherconfigured with one or more beamformers configured to measure signals inhorizontal or vertical spatial apertures. The hydrophone sensor array106 can be configured as a passive acoustic hydrophone sensor array oran active acoustic hydrophone sensor array with the inclusion of anadequate sound transmitter.

FIG. 1 further depicts a plurality of individual hydrophone sensors 108a-108 e, generally known as hydrophone sensors 108. Each individualhydrophone sensor is a hydrophone receiver capable of receiving acousticsignal data generated by targets of interest. Hydrophone sensors 108 canbe configured to operate (in the air as a microphone) at the surface(e.g., positively buoyant, floating or towed at the surface) as well asa neutrally-buoyant array (e.g., floating or towed below the surface),or fixed on the seabed. Hydrophone sensors 108 can includeomnidirectional, gradient, horizontal, as well as other directionallyoriented line array type receivers. Hydrophone sensors 108 with ahorizontal aperture (e.g. fixed or towed horizontal line arrays ordirectional sonobuoys), can calculate the bearing of an acoustic signalas well and therefore track the target of interest in near real time.The use of multiple hydrophone sensors 108 can produce a more accurateestimate of the location of the acoustic signal and target of interestthan an estimate of location produced by a single, individually deployedhydrophone sensor. Multiple hydrophone sensors can detect targets ofinterest in environments where a higher level of noise exists comparedto single hydrophone sensors because an array of sensors can utilizearray gain to rejection noise present in a received acoustic signal. Insome embodiments, an array as shown may operate at relatively lowfrequencies, e.g., DC through about 1 kHz. In some embodiments,multiples arrays may be used together, with each array being configuredto operate at a different range of operational frequencies. For example,a relatively high frequency array (HFA) such as a 5×5 Mill Cross Arrayoperational over a range of about 1 kHz to about 90 kHz (with a nominaldetection frequency of 50 kHz) can be used with the previously describedlow frequency array. Note: arrays may be configured for desired maximumdepths, e.g., 1000 m, 2000 m, etc. Additionally, or alternatively, asingle hydrophone sensor 108 can be utilized to perform autonomous jointdetection-classification. In such a single-hydrophone configuration, thesystem lacks the ability to spatially locate targets of interest inbearing without further triangulation of the acoustic signal with otherhydrophone sensors.

FIG. 1 includes a connection means 110 to link the hydrophone sensorarray 106 to the marine vehicle platform 102. In some implementations,the connection 110 can include a cable to tow the hydrophone sensorarray 106 behind the marine vehicle platform 102. The tow cable includesa strength member and a telemetry cable. The strength member can includea cable such as a nylon cable, Kevlar® cable, Spectra® cable, steelwire, or other high tensile strength cable to enable towing of thehydrophone sensor array 106 by the marine vehicle platform 102. Thetelemetry cable can include a cable such as a coaxial, serial, orEthernet cable, or the like, to facilitate or provide for transmissionof acoustic signal and target data from the hydrophone sensors 108 tothe signal processing system 104 included in the marine vehicle platform102.

FIG. 2 is a flowchart of an exemplary method of autonomous jointdetection-classification, and tracking of targets of interest that canbe implemented on a marine vehicle platform (e.g., the marine vehicleplatform 102 of FIG. 1). The method 200 includes receiving a pluralityof acoustic signals from two or more hydrophone sensors (stage 202) andjointly detecting and classifying a target of interest corresponding toeach of the received acoustic signals (stage 204). In someimplementations, the method 200 further includes generating a targettrack (stage 206) and further classifying each target track by analyzingthe spectrum of each target track (stage 208). In some implementations,the method 200 also includes outputting a compressed data reportidentifying the characteristics of the generated target track (210).

The method 200 includes receiving a plurality of acoustic signals fromtwo or more hydrophone sensors (stage 202). In some implementations, aplurality of acoustic signals can be received at two or more hydrophonesensors, such as hydrophone sensors 108. The acoustic signal isgenerated by a target of interest that is generating acoustic energywithin the range of detection of the hydrophone sensor 108. For example,a ship can be operating on the surface of the ocean within the range ofdetection of the hydrophone sensor array 106. The acoustic signalgenerated by the operation of the ship (e.g., the acoustic signalcreated by the ship's propeller rotating in the water) can be receivedby the two or more hydrophone sensors 108. In some implementations, theacoustic signal that is received by the two or more hydrophone sensors108 is transmitted autonomously and in real-time to the marine vehicleplatform 102 as input to the digital signal processing algorithmsimplemented in the signal processing system 104 that is configured inthe marine vehicle platform 102.

The method 200 also includes jointly detecting and classifying a targetof interest corresponding to each of the received acoustic signals(stage 204). In some implementations, the signal processing system 104included in the marine vehicle platform 102 can be configured withembedded processors storing computer readable instructions, which whenexecuted implement digital signal processing algorithms capable ofperforming autonomous joint detection-classification of a target ofinterest based on the acoustic signals received from the two or morehydrophone sensors 108. The signal processing algorithms can beimplemented to transform the received acoustic signals into spectralresponses in the frequency domain that are tested against the classifierhypothesis to determine a detection surface which spatially identifiesthe presence of a target of interest in bearing and time. The spectralresponse of a signal is the power spectrum of beamformer outputcorresponding to a given relative bearing. For background and details ofcertain known beamforming techniques, see, e.g., Kneipfer, R., “SonarBeamforming—An Overview of Its History and Status,” Naval UnderseaWarfare Center-NL Technical Report 10,003, (Apr. 7, 1992); and U.S. Pat.No. 6,980,486, both of which documents are incorporated herein byreference in their entireties. In some implementations, the spectralresponses can be integrated according to their frequency to compute adetection surface in bearing and time corresponding to the target ofinterest associated with the received acoustic signals. The subsequentassociation of detection events summarized by the decision surface isperformed to generate a target track. Additional details describing themethod of detecting a target of interest is presented in relation toFIG. 3.

As further shown in FIG. 2, the method includes generating a targettrack (stage 206). A target track is a time series of target radiatednoise (plus other ambient noise) signature along a target trackresulting from an inverse FFT being applied to a scissorgram. Ascissorgram is a time-frequency plot of the spectrum along a targettrack—the term “scissorgram” connotes the cutting and splicing ofinstantaneous spectrum slices from different beams as a function of timeas a target transits through beamspace. The digital signal processingalgorithms implemented in the signal processing system 104 can utilizethe detection surface calculated in stage 204 to track the location ofthe target of interest. In some implementations, a target track can becomputed by associating the spatial and spectral characteristics of eachdetection surface to produce a relative-bearing track as a function ofbearing and time. A relative-bearing track represents the locus ofdetection events that have been associated sequentially into a group ofpoints that estimate the bearing history of a detected target. Therelative-bearing track can be reconciled with an estimate of theorientation of the hydrophone sensor array to calculate a true bearingof the acoustic signal source which can be used to track the position ofthe target of interest. Additional details describing the method oftracking the location of a target of interest are presented in relationto FIG. 4.

The method 200 further includes generating a summary report of spectraldata associated with each target track (stage 208) and outputting acompressed data report identifying the spectral characteristics of thegenerated target track (stage 210), such as those present in thescissorgram. In some implementations, the summary report can include anynumber of user-defined or user-selected parameters associated with thespectral characteristics of each generated target track. Additionally oralternatively, the summary report can include any uniquely identifyingspectral characteristics of the detected target's radiated noisespectrum (e.g. fundamental frequency, harmonic indices, highest SNRfeature, etc.). The compressed data report can include any data that ispresent in the generated summary report as well as characteristics ofthe target track including, but not limited to, time-stamp data, marinevehicle platform position or location data, relative bearing of acousticsignal source, the key identifying frequency of the acoustic signalgenerated by the source, the classifier type, the target of interestclassification type (e.g., a vessel or ship, a marine mammal, or ageological phenomenon) and a classifier confidence score. The confidencescore can be represented as probability or percentage. The confidencescore is intended to characterize the accuracy or likelihood that thetarget of interest that has been classified by the system is identifiedas the actual or correct target of interest type. In someimplementations, the compressed data report can be a concise reportconfigured for output as a minimally sized file or data structure (e.g.,100 bytes or less). The small size of the compressed data report or filecan facilitate data or file transmission in operating environments orover communication channels where bandwidth is limited. Additionally, oralternatively, the signal processing system 104 (shown in FIG. 1) can beconfigured to generate and output the compressed data report atpre-determined time intervals, such as once every five seconds, onceevery minute, once every hour, etc.

FIG. 3 is a flowchart of a method 300 of autonomous jointdetection-classification of a target of interest corresponding toreceived acoustic signals. The method 300 includes transforming thereceived acoustic signal from the time domain to the frequency domain(stage 302). The method also includes generating a relative-bearing beamresponse for the transformed signals in one or more steering directions(stage 304). The method also includes estimating the median backgroundnoise level of the relative-bearing beam response in each of the one ormore steering directions (stage 306) and normalizing the spectralresponse of the relative-bearing beam response by the estimated noisebackground level (stage 308). The method further includes performing ajoint detection-classification operation (stage 310) and computing adetection surface in bearing and time based on the frequency integrationof the join detection-classification operation (stage 312). The methodalso includes determining a decision surface in bearing and time bycalculating a constant-false-alarm-rate (CFAR) detection threshold fromthe estimated background noise level and applying theconstant-false-alarm-rate (CFAR) detection threshold to the detectionsurface (stage 314).

The method 300 includes transforming a received acoustic signal from thetime domain to the frequency domain (stage 302, with “stage” includingreference to a software module where context permits). The receivedacoustic signals are transformed using a Fourier transform, such as afast Fourier transform (FFT). The method 300 also includes generating arelative-bearing beam response for the transformed signals in one ormore steering directions (stage 304). For example, the received acousticsignals can be transformed using frequency domain or time domainbeamforming to generate a relative-bearing or conical beam response inmultiple beam steering directions. The relative-bearing beam response isthe output of the beamformer as a function of bearing for a givenfrequency. The relative-bearing beam response is determined bymultiplying an M×N beamformer steering matrix (where M represents thenumber of beams and N represents the number of sensors in the acousticarray) with an N×1 hydrophone sensor array measurement resulting in aM×1 complex vector of beamformer output response. The beam steeringdirection is the relative bearing angle associated with a givenbeamformer steering vector. The beamforming steering vector is an N×1column vector of complex spatial filter coefficients (e.g., one columnof the N×M steering matrix mentioned above) that is conjugate-transposedand multiplied against an N×1 element hydrophone array vector snapshotto yield a complex, scalar-valued beam response for a given relativebearing. The magnitude-squared of this complex value is the beamformeroutput power at this bearing. In some implementations, the acousticsignals can be transformed to generate a relative-bearing beam responsein one or more beam steering directions spanning the azimuthal spacefrom forward endfire to aft endfire. In other implementations, thespatial filtering operation can include a linear or conventional methodfor computing the spatial filter weighting coefficients. In someimplementations, the spatial filtering operation can include a dataadaptive method for computing the spatial filter weighting coefficientsof the beamformer required to generate the desired relative bearing beamresponse.

In some implementations, the acoustic signal is conditioned prior toperforming the spatial filtering operation. For example, the receivedacoustic signals can be automatically conditioned to detect and removebias from the signal. In other implementations, the received acousticsignal can be conditioned to shade an inoperative channel. In someimplementations, the received acoustic signal can be conditioned fortransient suppression or to perform sub-aperture phase alignment of theacoustic signal prior to further processing. In other implementations,the received acoustic signal can be conditioned to share all channelsfor suppression of sidelobes in the beamformer output.

In some implementations, a three-dimensional soundscape can be createdbased on the spatial filtering operations. The three-dimensionalsoundscape can quantify the distribution of sound in the horizontal andvertical bearing directions as associated with the target track or timeseries data corresponding to each acoustic signal.

The method 300 further includes estimating the median background noiselevel of the relative-bearing beam response in each of the one or moresteering directions (stage 306). In some implementations, the noisespectrum of the relative-bearing beam responses is equalized byestimating the median background noise level in each steering direction.The method 300 also includes normalizing the spectral response of therelative-bearing beam response by the estimated noise background level(stage 308). In some implementations, a frequency-dependent,relative-bandwidth, median-filter based spectrum normalizer is utilizedto normalize the spectral response of the relative-bearing beamresponses. For example, the spectrum normalizer as described, providesimproved noise spectrum whitening performance compared to other spectrumnormalizers due to its ability to more accurately estimate and removetemporal fluctuations in the dynamic background noise level. Thisimproved functionality is particularly useful in cluttered or high-noisemarine environments for detecting low frequency acoustic signals wherebackground noise spectral coloring can vary rapidly with time andfrequency.

The method 300 also includes performing a joint detection-classificationoperation (software module, or stage, 310). A jointdetection-classification operation is performed consisting of aselective frequency integration of the normalized spectral responsesassociated with each relative-bearing beam response such that thefrequency integration is informed, filtered, or constrained by thedetailed physics of the underlying classifier for the target of interestto satisfy a spatial and/or spectral target hypothesis of theclassifier. The classifier indicates radiated noise characteristics orphysics, such as the narrowband tonal set membership of a harmonicfingerprint for example, that are output from a classifier associatedwith the target of interest. For example, by utilizing the detailedphysics data that is output from the classification algorithm inconjunction with the highly-selective frequency integration ensures thatonly the energy best satisfying the spatial and spectralhypothesis/hypotheses (assumptions) of the classifier is permitted to beused in the frequency integration. In this way, the frequencyintegration is enhanced, or informed by the radiated noisecharacteristics of the target yielding a summed relative-bearing beamresponse that exhibits maximal signal-to-noise ratio, superior spatialresolution, and is uniquely associated with the acoustic source ortarget of interest described by the classifier algorithm. This techniquemore accurately differentiates the acoustic signal associated with atarget of interest from the presence of noise in a received acousticsignal resulting in increased detection sensitivity in regard to theacoustic signal source of interest. A wide variety of classifiers(classification algorithms or libraries of a priori acoustic data, e.g.,harmonic signatures for a particular acoustic source of interest) can beused; a specific classifier can be chosen based on the desired target ofinterest to be detected, classified and tracked. In someimplementations, multiple classifiers can be used, simultaneously orsequentially, to detect and simultaneously classify multiple targets ofinterest.

The method 300 includes computing a detection surface in bearing andtime based on the frequency integration of the jointdetection-classification operation (stage 312). A detection surface is atwo-dimensional bearing vs. time record (BTR) plot of thefrequency-integrated relative bearing beam response showing thedistribution of acoustic energy in bearing as a function of time. TheBTR is the most commonly used as a scene awareness tool for summarizingthe distribution of acoustic contacts. In some implementations, afrequency integration of the normalized spectral responses can beperformed to compute a detection surface. In some implementations, thedetection surface can be represented in bearing and time and can beassociated with each relative-bearing beam response. In someimplementations, the detection surface is computed using an advanceddetection method which mitigates the sidelobe response of strongdiscrete sources of acoustic interference prior to the frequencyintegration of the normalized spectral response associated with eachrelative-bearing beam response.

The method includes determining a decision surface in bearing and timeby calculating a constant-false-alarm-rate detection (CFAR) thresholdfrom the estimated noise background level and applying theconstant-false-alarm-rate (CFAR) detection threshold to the detectionsurface (stage 314). A decision surface is a thresholded detectionsurface, or a BTR of those detection events or pixels in the detectionsurface that exceeded the detection threshold. Pixels on the decisionsurface are the detection events that get associated into target tracksassociated with a target. In some implementations, aconstant-false-alarm-rate (CFAR) thresholding step can be calculatedfrom the estimated noise background level and applied to the detectionsurface. The constant-false-alarm-rate (CFAR) detection threshold can beapplied on a temporal scan-by-scan basis to the sidelobe adjusteddetection surface to determine a corresponding decision surface inbearing and time corresponding to the detected target of interest. Thesensitivity of the determined constant-false-alarm-rate (CFAR) detectionthreshold is increased due to the inherent screening potential of thespatial-spectral set membership when a parallel or jointdetection-classification system and method are implemented as describedherein. The benefit of increased constant-false-alarm-rate detectionthreshold sensitivity includes improvements to important performancecharacteristics such as enhanced target of interest hold time ratio aswell as improved probability of correct classification for a fixed falsealarm rate. For example, the joint detection-classification system andmethod only renders and outputs (for example, via a compressed datareport) targets of interest that are consistent with the classifiercriteria in the determined constant-false-alarm-rate (CFAR) detectionthreshold.

FIG. 4 is a flowchart of a method 400 for generating a target trackcorresponding to the decision surface. The method includes associatingdecision surface threshold exceedances to produce a relative-bearingtrack as a function of bearing and time (stage 402). The method alsoincludes calculating a true bearing track corresponding to each of therelative-bearing tracks by reconciling the relative-bearing track withan estimate of hydrophone sensor array orientation (stage 404). Themethod further includes transforming the received acoustic signal fromthe frequency domain to the time domain using the relative-bearing trackto generate a target track (stage 406).

The method 400 includes associating decision surface thresholdexceedances to produce a relative-bearing track as a function of bearingand time (stage 402). In other implementations, a spatial trackingoperation can be performed to associate instantaneous detectiondecisions on the basis of spectral proximity or similarity. Asimplemented in the joint detection-classification system and method, thespatial bandwidth of the association algorithm is permitted to vary withfrequency, accounting for the fact that the beam response of the sensorarray scales with the wavelength of the acoustic signal. For example,lower frequency features are afforded wider spatial tolerance thanhigher frequency ones. This has the effect of enhanced main lobesensitivity and, thus, reduced susceptibility to instantaneous falsealarms. In some implementations, a spatial tracking operation can beperformed to associate instantaneous detection decisions on the basis ofspatial proximity or similarity.

The method 400 includes calculating a true bearing track correspondingto each of the relative-bearing track by reconciling therelative-bearing track with an estimate of hydrophone sensor arrayorientation (stage 404). In some implementations, a unique estimate ofthe true bearing track can be determined by reconciling therelative-bearing track with an estimate of the instantaneousfluctuations in the orientation or position of the hydrophone sensorarray 106.

The method 400 includes transforming the received acoustic signals fromthe frequency domain to the time domain using the relative-bearing trackto generate a target track (stage 406). In some implementations aninverse fast Fourier transform (IFFT) algorithm is applied to thereceived acoustic signals or conditioned acoustic signals. In someimplementations, the IFFT algorithm uses the relative-bearing track tocompute a digital beam response time series. The computed digital beamresponse time series represents a target track corresponding to eachtarget of interest. Additionally, or alternatively, a scissorgram can begenerated and exported for any generated target track. A scissorgram isa spectrogram, computed from the target track, plotting the frequency ofthe target's acoustic signal as a function of time. In someimplementations, the method can generate a target track for furtherclassification.

FIG. 5 is a diagram of an exemplary system configuration for receivingacoustic signals using a marine vehicle platform. In broad overview, theexemplary system configuration 500 includes one or more individualhydrophone sensors, for example a single hydrophone sensor such ashydrophone sensor 505N as shown in brackets or two or more hydrophonesensors 505A-505E, generally referred to as hydrophone sensors 505. Theplurality of hydrophone sensors 505 are attached by a connection meanssuch as a tow/data cable 510 to the marine vehicle platform 515. Themarine vehicle platform 515 includes a detection, classification, andtracking system 520 to receive the transmitted acoustic signals in orderto perform autonomous joint detection-classification and tracking inreal-time on the received acoustic signals. The marine vehicle platform515 further includes a GPS receiver 525. The detection, classification,and tracking system 520 includes an FFT Module 535 to receive acousticsignals 530 from the hydrophone sensors 505 and transform them to thefrequency domain. The detection, classification, and tracking system 520also includes beamforming module 540, spectrum normalizer module 545 andjoint detector-classifier 550. The joint detector-classifier 550includes classifier module 560 and spatial-spectral membership detector555. The detection, classification, and tracking system 520 alsoincludes track assignment module 565, true bearing solver module 570 andalert generator module 575. The marine vehicle platform may also includea communications (“comms”) module 580 to output acoustic signal andtarget data for use elsewhere at locations remotely situated from themarine vehicle platform 515.

Referring to FIG. 5 and FIG. 2, a plurality of hydrophone sensors 505can be interconnected to form an array of hydrophone sensors. The arrayof hydrophone sensors can include one or more acoustic sensors, e.g.,individual hydrophone sensors 505, interconnected by cables or otherattachment means linking individual hydrophone sensors 505 to form ahydrophone sensor array, for example, such as hydrophone sensor 106described in relation to FIG. 1. Each hydrophone sensor 505 includes ahydrophone receiver (not shown) which acts as microphone to recordacoustic signals in marine environments. In some implementations, thearray of hydrophone sensors 505 can include multi-channel hydrophonesensors, linear hydrophone sensors, planar hydrophone sensors, mobilehydrophone sensors, and/or fixed hydrophone sensors. Additionally, oralternatively, the hydrophone sensors included in the array ofhydrophone sensors 505 can each include an analog-to-digital converterto process the received acoustic signal and transform the receivedsignal to digital signal data for transmission to the detection,classification and tracking system 520. In some implementations, thehydrophone sensor array 505 can further include non-acoustic sensorssuch as sensor components to determine position or orientation data ofthe array such as heading, pitch, roll and depth.

As shown in FIG. 5, the collection of hydrophone sensors 505 configuredas a hydrophone sensor array are linked or joined by connection orattachment means 510 to marine vehicle platform 515. In someimplementations, a cable is used to link the collection of hydrophonesensors 505, formed as an array, to the marine vehicle platform 515. Insome implementations, the hydrophone sensor array can be towed by themarine vehicle platform 535. In other implementations, a data ortelemetry cable may connect the hydrophone sensor array to the marinevehicle platform 515. For example, a coaxial, serial or Ethernet datacable can be configured to transmit digitized acoustic signals from thehydrophone sensor array 505 to the marine vehicle platform 515. In someimplementations, the hydrophone sensor array is external to the marinevehicle platform 515. In other implementations, the hydrophone sensorarray is mounted to the marine vehicle platform 515. For example, ahydrophone sensors array may be mounted to the hull of an autonomousmarine vehicle platform. The hydrophone sensor array may also bedeployed (as a microphone sensor array) in air or above the surface ofthe marine environment.

Still referring to FIG. 5, the marine vehicle platform 515 can includeautonomous or unmanned vehicles or vehicle platforms. The marine vehicleplatform 515 can include submersible or surfaced vehicles or vehicleplatforms. The marine vehicle platform 515 is similar to marine vehicleplatform 102 described in relation to FIG. 1. The marine vehicleplatform 515 can operate autonomously and can be propelled using anelectric-motor driven propeller or a buoyancy-driven propulsion system.In some implementations, the vehicle platform may be a terrestrial oraerial vehicle platform. The vehicle platform may be a non-mobile,stationary platform. Additionally, or alternatively the vehicle platformcan be remotely operated by an operator situated away from the locationof the vehicle.

As shown in FIG. 5, the marine vehicle platform 515 further includes adetection, classification, and tracking system 520. For example, thedetection, classification, and tracking system 520 can be included insignal processing system 104 as shown in FIG. 1. The detection,classification, and tracking system 520 receives acoustic signals 530from hydrophone sensors 505 via a data or telemetry cable included inconnection means 510. The detection, classification, and tracking system520 includes a plurality of embedded processors configured withspecialized digital signal processing algorithms to perform autonomouslyjoint detection-classification, and tracking of a target of interestcorresponding to each of the respective acoustic signals 530 receivedfrom the plurality of hydrophone sensors 505.

As further shown in FIG. 5, the marine vehicle platform 515 furtherincludes a GPS receiver 525. The GPS receiver 525 accurately tracks theprecise position of the marine vehicle platform 515 while it is at thesurface. GPS data can be included in the compressed data report that isoutput from the detection, classification, and tracking system 520.

Referring to FIG. 5, the detection, classification, and tracking system520 is configured to receive a plurality of acoustic signals 530transmitted from hydrophone sensors 505. The transmitted acousticsignals 530 are received at the FFT module 535 and transformed from thetime domain to the frequency domain. In some implementations, prior totransforming the acoustic signals from the time domain to the frequencydomain, the acoustic signals are conditioned for bias detection andremoval, inoperative channel shading, transient suppression, sidelobesuppression, and/or sub-aperture phase alignment. In someimplementations, the received acoustic signals are recorded by a digitalelement recorder (not shown). The time series data associated with eachacoustic signal 530 can be continuously recorded by the element recorderand post-processed at a later time.

As shown in FIG. 5, the detection, classification, and tracking system520 also includes a beamformer module 540. The beamformer module 540 isa spatial filter configured to select acoustic sources in a givendirection while suppressing acoustic sources in all other directions.The beamformer module 540 can generate a relative-bearing beam responsefor each of the transformed signals in one or more beam steeringdirections. In some implementations, the beamformer module 540 is afrequency domain beamformer. In other implementations, generatingrelative bearing beam response in one or more beam steering directionsalso includes linear or data-adaptive methods for computing the spatialfilter weighting coefficients of each relative-bearing beam response.

As further shown in FIG. 5, the detection, classification, and trackingsystem 520 also includes a spectrum normalizer module 545. The spectrumnormalizer module 545 can be a frequency-dependent, relative-bandwidth,median-filter based spectrum normalizer. The spectrum normalizer module545 estimates the median background noise level of the relative-bearingbeam response in each of the one or more beam steering directions andnormalizes the spectral response of each relative-bearing beam response.

As shown in FIG. 5, the detection, classification, and tracking system520 also includes a joint detector-classifier 550. The jointdetector-classifier 550 includes a spatial-spectral set membershipdetector 555 and a classifier module 560. The spatial-spectralmembership detector 555 performs frequency integration on the normalizedspectral responses associated with each relative-bearing beam responseby utilizing radiated noise characteristics and data output fromclassifier module 560.

The joint detector-classifier 550 shown in FIG. 5 further includesclassifier module 560. Classifier module 560 includes or presents aclassifier describing a priori and/or learned knowledge of the physicsof a target of interest. For example, the classifier in the classifiermodule can be an algorithm or model describing the acoustic signature ofa target of interest. The classifier module 560 can be a classifierlibrary storing the radiated noise characteristics or attributes of avariety of different targets of interest. In some implementations, theclassifier 560 may include attributes associated with a target ofinterest such as the range, depth, relative and/or true bearing, course,speed, frequency, range rate and/or whether the target of interest islocated at the surface or submerged. Additionally, or alternatively, theclassifier 560 may also include the fundamental frequency (of aharmonically related set of frequencies), Doppler (in the case of anactive sonar return), and/or species identification data associated witha target of interest. For example, classifier module 560 can includeclassification data or profiles associated with recreational orcommercial vessels, and marine wildlife such as whales, fish, or marinemammals. Additionally or alternatively, classifier module 560 mayinclude classification data or profiles associated with geologicalphenomenon, such as hydrothermal vents or underwater volcanic activity.The classification data or profiles identify specific characteristics ofthe physics associated with the acoustic signal radiated from a targetof interest. The classifier module 560 provides acoustic data associatedwith the desired target of interest as input to the spatial-spectral setmember ship detector 555 in order to classify the target of interest inparallel with the frequency integration. In some implementations,classifier module 560 can employ harmonic fingerprint classifiers,linear classifiers, quadratic classifiers or kernel estimationtechniques to accurately classify each target track representing atarget of interest. The joint detection-classification system and methodare configured such that the entire detection-classification string isimplemented at the spectral resolution required by the most demandingclassification algorithm.

As further shown in FIG. 5, the detection, classification, and trackingsystem 520 also includes a track assignment module 565. The trackassignment module 565 can compute a detection surface in bearing andtime based on the frequency integration. In some implementations, themethod of computing the detection surface includes mitigating thesidelobe response of strong discrete sources of acoustic interferenceprior to the frequency integration of the normalized spectral responseassociated with each relative bearing beam response. The trackassignment module 565 can calculate a constant-false-alarm-rate (CFAR)detection threshold based on applying the estimated noise backgroundlevel to the detection surface and determine a decision surface inbearing and time based on the calculated constant-false-alarm-rate(CFAR) detection threshold resulting in a relative-bearing track of thetarget.

Still referring to FIG. 5, the detection, classification, and trackingsystem 520 also includes a true bearing solver module 570 to generate atrue bearing target track corresponding to each decision surface. Thetrue bearing solver module 570 determines which decision surfacesexceeding the constant-false-alarm-rate (CFAR) detection threshold canbe associated based on spatial and spectral characteristics and producesa relative-bearing track as a function of bearing and time. The truebearing solver module 570 calculates a true bearing of therelative-bearing track by reconciling the relative-bearing tracks withan estimate of hydrophone sensor array orientation. The true bearingsolver module 570 uses the decision surface to transform the acousticsignals from the frequency domain to the time domain in order togenerate a target track reflecting the true location in bearing and timeof the detected target of interest. In some implementations, thetransformation of the acoustic signals from the frequency domain to thetime domain is performed using an inverse Fast Fourier Transformalgorithm.

As shown in FIG. 5, the detection, classification, and tracking system520 includes alert generator module 575. In some implementations, thealert generator module 575 can generate a summary report of spectraldata associated with each target track representing a target ofinterest. The alert generator module 575 can output a compressed datareport identifying the characteristics of the generated target trackincluded in the summary report using a communication module, such ascomms module 580 (shown in dashed lines to indicate it is separate fromthe detection, classification, and tracking system 520). In someimplementations, comms module 580 may be included as a communicationmodule associated with the vehicle platform, such as marine vehicleplatform 515. For example, comms module 580 can be an acoustic modem. Inother implementations, comms module 580 can be a small-footprint,low-power modem or micromodem capable of transmitting data at rates of80-5400 bits per second. Additionally, or alternatively, the commsmodule 580 can include means for communicating via satellitecommunication, such using the Iridium satellite constellation. Dependingon the configuration of the vehicle platform, the vehicle may berequired to surface to transmit the compressed data report. In someimplementations, the compressed data report is configured to be aminimal data size to facilitate transmission over bandwidth limitedcommunication channels or in bandwidth limited operating environments.For example, the compressed data report file size can be configured toinclude a set of alert reports, each of which does not exceed 100 bytes.The alert generator module 575 can output the compressed data report viacomms module 580 to a receiver situated remotely from the vehicleplatform, such as marine vehicle platform 515. Additionally, oralternatively, the compressed data report can be user-configured oruser-defined to specify the type of data associated with the target ofinterest and the received acoustic signal to be included in thecompressed data report.

In some implementations, the detection, classification, and trackingsystem 520 is further configured to measure the location of a target ofinterest and allow later quantification of parameters associated witheach generated target track including detection probability, detectionfalse alarm probability, classification probability, classificationfalse alarm probability, as well as system processing gain and bearingerrors.

In some implementations, a system can be configured in a calibrationmode as a separate surrogate target that is used to tune the detection,classification and tracking functionality performed by the embeddedprocessors. For example, the system can be configured to transmit atime-synchronized acoustic signal that can be received by the hydrophonesensors 505. The use of a time-synchronized signal enables the system toaccurately determine the precise bearing and frequency of the target ofinterest and tune the joint detector-classifier 550 to achieve increaseddetection and classification performance with a decreased classificationfalse alarm rate. The surrogate system can be configured with differenttarget of interest profiles (e.g., varying amplitude and frequencycharacteristics) to be transmitted and thereby enabling the jointdetection-classification performance to be evaluated in regard to knowntargets of interest. Exemplary techniques and systems for suitablecalibration are described in co-owned U.S. Pat. No. 7,760,587B2, whichis incorporated herein by reference in its entirety.

FIGS. 6A and 6B are diagrams demonstrating the calculation of anacoustic signal's frequency-integrated beam response by conventionalmethods and by joint detection-classification methods as describedherein. As shown in FIGS. 6A and 6B, a given acoustic signal may bereceived by each hydrophone sensor in a hydrophone sensor array. Thehydrophone sensors in the array each detect the signal and a beamresponse is generated by the detection, classification, and trackingsystem 520. The beam responses may be frequency-integrated or summed togenerate a relative-bearing beam response as a function of time. Asshown in FIGS. 6A and 6B, each beam response is represented as a plot ofspectral energy in terms of amplitude and frequency. For example, Beam Nis an acoustic signal including three distinct, harmonically relatedpeaks (or related in some other manner as described by the classifieralgorithm) occurring at frequencies f1, f2 and f3, respectively. As themarine vehicle platform towing the hydrophone sensor array navigatesthrough the water, the acoustic signal is received by the plurality ofhydrophone sensors in the array. For example, Beam 1 is therelative-bearing beam response generated by the detection,classification, and tracking system 520 in a particular direction. Therelative beam response in the direction nearest to the target ofinterest emitting the acoustic signal will be the beam response with thegreatest amplitude. As shown in FIGS. 6A and 6B, the beam response isgreatest in Beam K. The amplitude of the beam responses associated withBeams K−1 and K+1 are less because the directions of the beam responsesare father away from the target of interest emitting the acousticsignal. Methods and systems for calculating and displaying sonar dataare described in U.S. Patent Publication No. US 2003/0227823A1, which isincorporated herein by reference in its entirety

As shown in FIG. 6A, conventional methods of acoustic signal detectionand classification sum the beam responses over all frequencies to reportthe level of the summed beam response relative to the bearing angle ofthe acoustic signal. In this manner, the summed beam responses are notvery sensitive to the direction of the targets of interest and may leadto missed detections in noisy environments.

As shown in FIG. 6B, the joint detection-classification method sums thebeam responses only over the frequencies that best bit the classifierhypothesis. Additionally, these frequencies are integrated only for thebeam response at which they are maximal, e.g., Beam K. For example, byperforming frequency integration utilizing the radiated noisecharacteristics output from a classifier associated with a given targetof interest in parallel only the frequencies at one bearing moststrongly correlated with the classifier output are included in thesummed beam response. As described, the joint detection-classificationmethod can result in more precise estimation of the direction associatedwith the detected acoustic signal (e.g., the target of interest).Systems and methods configured with the joint detection-classificationfunctionality described herein enable acoustic signals to be classifiedwith greater accuracy in the presence of higher noise levels and withdecreased false alarm rates such that the overall system and method aremore robust compared to conventional techniques for acoustic signaldetection.

FIG. 7 is a block diagram of a computing system 710 suitable for use inimplementing the computerized components described herein. In broadoverview, the computing system 710 includes at least one processor 750for performing actions in accordance with instructions, and one or morememory devices 760 and/or 770 for storing instructions and data. Theillustrated example computing system 710 includes one or more processors750 in communication, via a bus 715, with memory 770 and with at leastone network interface controller 720 with a network interface 725 forconnecting to external devices 730, e.g., a computing device (such as ahydrophone digitizer). The one or more processors 750 are also incommunication, via the bus 715, with each other and with any I/O devicesat one or more I/O interfaces 740, and any other devices 780. Theprocessor 750 illustrated incorporates, or is directly connected to,cache memory 760. Generally, a processor will execute instructionsreceived from memory.

In more detail, the processor 750 can be any logic circuitry thatprocesses instructions, e.g., instructions fetched from the memory 770or cache 760. In many embodiments, the processor 750 is an embeddedprocessor, a microprocessor unit or special purpose processor. Thecomputing system 710 can be based on any processor, e.g., suitabledigital signal processor (DSP), or set of processors, capable ofoperating as described herein. For example, suitable implementations ofprocessor 750 can include processors such as the included in an ODROIDplatform (e.g., the ODROID-XU4, ODRIOD-C2, and/or ODROID-C1+) as madecommercially available by the Hardkernel Co., Ltd. of South Korea; othersuitable processors and processor platforms may be used. In someembodiments, the processor 750 can be a single core or multi-coreprocessor. In some embodiments, the processor 750 can be composed ofmultiple processors.

The memory 770 can be any device suitable for storing computer readabledata. The memory 770 can be a device with fixed storage or a device forreading removable storage media. Examples include all forms ofnon-volatile memory, media and memory devices, semiconductor memorydevices (e.g., EPROM, EEPROM, SDRAM, flash memory devices, and all typesof solid state memory), magnetic disks, and magneto optical disks. Acomputing device 710 can have any number of memory devices 770.

The cache memory 760 is generally a form of high-speed computer memoryplaced in close proximity to the processor 750 for fast read/writetimes. In some implementations, the cache memory 760 is part of, or onthe same chip as, the processor 750.

The network interface controller 720 manages data exchanges via thenetwork interface 725. The network interface controller 720 handles thephysical and data link layers of the Open Systems Interconnect (OSI)model for network communication. In some implementations, some of thenetwork interface controller's tasks are handled by the processor 750.In some implementations, the network interface controller 720 is part ofthe processor 750. In some implementations, a computing device 710 hasmultiple network interface controllers 720. In some implementations, thenetwork interface 725 is a connection point for a physical network link,e.g., an RJ 45 connector. In some implementations, the network interfacecontroller 720 supports wireless network connections and an interfaceport 725 is a wireless receiver/transmitter. Generally, a computingdevice 710 exchanges data with other network devices 730, such ascomputing device 730, via physical or wireless links to a networkinterface 725. In some implementations, the network interface controller720 implements a network protocol such as Ethernet.

The other computing devices 730 are connected to the computing device710 via a network interface port 725. The other computing device 730 canbe a peer computing device, a network device, or any other computingdevice with network functionality. For example, a computing device 730can be a digital hydrophone sensor or hydrophone digitizer, an array ofdigitized hydrophone sensors or a network device such as a hub, abridge, a switch, or a router, connecting the computing device 710 to adata network such as the Internet.

In some uses, the I/O interface 740 supports an input device and/or anoutput device (not shown). In some uses, the input device and the outputdevice are integrated into the same hardware, e.g., as in a touchscreen. In some uses, such as in a server context, there is no I/Ointerface 740 or the I/O interface 740 is not used. In some uses,additional other components 780 are in communication with the computersystem 710, e.g., external devices connected via a universal serial bus(USB).

The other devices 780 can include an I/O interface 740, external serialdevice ports, and any additional co-processors. For example, a computingsystem 710 can include an interface (e.g., a universal serial bus (USB)interface, or the like) for connecting input devices (e.g., a keyboard,microphone, mouse, or other pointing device), output devices (e.g.,video display, speaker, refreshable Braille terminal, or printer), oradditional memory devices (e.g., portable flash drive or external mediadrive). In some implementations an I/O device is incorporated into thecomputing system 710, e.g., a touch screen on a tablet device. In someimplementations, a computing device 710 includes an additional device780 such as a co-processor, e.g., a math co-processor that can assistthe processor 750 with high precision or complex calculations.

The components, steps, features, benefits, and advantages which havebeen discussed are merely illustrative. None of them, or the discussionsrelating to them, are intended to limit the scope of protection in anyway. Numerous other embodiments are also contemplated. These includeembodiments which have fewer, additional, and/or different components,steps, features, objects, benefits and advantages. These also includeembodiments in which the components and/or steps are arranged and/orordered differently.

For example, while the foregoing description has been provided in thecontext of processing acoustic data, related to under-water targets orsources of interest, that is received from under-water acoustic sensorsand sensor arrays, systems and methods according to the presentdisclosure can also provide for similar processing of acoustic data innon-marine environments, e.g., where acoustic data related to acousticsources of interest (in air) is received from acoustic arrays that arenot submerged.

In some implementations, the system and method described herein can beused to perform autonomous joint detection-classification, and/ortracking of acoustic sources located outside of the water (liquid). Forsuch embodiments the hydrophone sensor array is replaced by one or moresuitable acoustic sensor arrays, e.g., a microphone sensor array,configured to collect acoustic data for another sound-conveying medium,e.g., air, or the Earth (for seismic data), etc. In one example, amicrophone array may be configured on a marine vehicle platform (whichcould also have or be linked to one or more sub-surface hydrophonearrays as described previously). The marine vehicle platform can surfaceto position the microphone sensor array above the water such thatacoustic signals generated from targets of interest above or outside ofthe water can be autonomously processed according to the jointdetection-classification and tracking system and method of the presentdisclosure. For example, the marine vehicle platform can operate in thelittoral zone of a body of water in order to detect, classify and trackacoustic signals from targets of interest located near, but not residingin, the littoral zone. The marine vehicle platform can position itselfsuch that the attached microphone sensor array can receive acousticsignals originating outside of the littoral zone of the water, such asacoustic signals originating from terrestrial targets of interest thatare situated on land in close proximity to the water. In otherembodiments, a single acoustic sensor can be used alone for jointdetection-classification of one or more acoustic sources of interested,without performing a step of tracking.

Implementations of the subject matter and the operations described inthis specification can be implemented in digital or analog electroniccircuitry, or in computer software embodied on a tangible medium,firmware, or hardware, including the structures disclosed in thisspecification and their structural equivalents, or in combinations ofone or more of them. Implementations of the subject matter described inthis specification can be implemented as one or more computer programsembodied on a tangible medium, i.e., one or more modules of computerprogram instructions, encoded on one or more computer storage media forexecution by, or to control the operation of, a data processingapparatus. In exemplary embodiments, the C programming language can beused; in other embodiments, different suitable programming languages canbe used, including, but not limited to: C++, C#, PASCAL, FORTRAN,MATLAB, Octave, Scilab, Julia, VHDL, Verilog, System-C, or the like. Acomputer storage medium can be, or be included in, a computer-readablestorage device, a computer-readable storage substrate, a random orserial access memory array or device, or a combination of one or more ofthem. The computer storage medium can also be, or be included in, one ormore separate components or media (e.g., multiple CDs, disks, or otherstorage devices). The computer storage medium can be tangible andnon-transitory.

The operations described in this specification can be implemented asoperations performed by a data processing apparatus on data stored onone or more computer-readable storage devices or received from othersources.

While this specification contains many specific implementation details,these should not be construed as limitations on the scope of anyinventions or of what can be claimed, but rather as descriptions offeatures specific to particular implementations of particularinventions. Certain features that are described in this specification inthe context of separate implementations can also be implemented incombination in a single implementation. Conversely, various featuresthat are described in the context of a single implementation can also beimplemented in multiple implementations separately or in any suitablesub-combination. Moreover, although features can be described above asacting in certain combinations and even initially claimed as such, oneor more features from a claimed combination can in some cases be excisedfrom the combination, and the claimed combination can be directed to asub-combination or variation of a sub-combination.

References to “or” can be construed as inclusive so that any termsdescribed using “or” can indicate any of a single, more than one, andall of the described terms. The labels “first,” “second,” “third,” andso forth are not necessarily meant to indicate an ordering and aregenerally used merely to distinguish between like or similar items orelements.

Thus, particular implementations of the subject matter have beendescribed. Other implementations are within the scope of the followingclaims. In some cases, the actions recited in the claims can beperformed in a different order and still achieve desirable results. Inaddition, the processes depicted in the accompanying figures do notnecessarily require the particular order shown, or sequential order, toachieve desirable results. In certain implementations, multitasking orparallel processing can be utilized.

What is claimed is:
 1. A method for autonomous jointdetection-classification and tracking of targets of interest, the methodcomprising: receiving a plurality of acoustic signals from a pluralityof acoustic sensors, each of the respective acoustic signalscharacterized by time-series data; detecting a target of interest fromthe received acoustic signals by: (i) transforming the received acousticsignals from the time domain to the frequency domain; (ii) generating arelative-bearing beam response for each of the transformed signals inone or more beam steering directions; (iii) estimating the medianbackground noise level of the relative-bearing beam response in each ofthe one or more beam steering directions and normalizing the spectralresponse of the relative-bearing beam response by the estimatedbackground noise level, wherein a normalized spectral response isproduced that is associated with each relative-bearing beam response;(iv) performing a joint detection-classification operation, wherein thejoint detection-classification operation comprises a selective frequencyintegration of the normalized spectral response associated with eachrelative-bearing beam response, wherein the frequency integration isinformed by the detailed physics of the underlying classifier for thetarget of interest to satisfy a spatial and spectral target hypothesisof the classifier; (v) computing a detection surface in bearing and timebased on the frequency integration of the joint detection-classificationoperation; (vi) determining a decision surface in bearing and time bycalculating a constant-false-alarm-rate (CFAR) detection threshold fromthe estimated noise background level and applying theconstant-false-alarm-rate detection threshold to the detection surface;and (vii) associating decision surface threshold exceedances with thetarget, and thereby detecting the target.
 2. The method of claim 1,wherein the plurality of acoustic sensors includes a microphone.
 3. Themethod of claim 1, further comprising generating a target rackcorresponding to the decision surface by: (i) associating decisionsurface threshold exceedances to produce a relative-bearing track as afunction of bearing and time; (ii) calculating a true bearing trackcorresponding to each of the relative-bearing tracks by reconciling therelative-bearing tracks with an estimate of hydrophone sensor arrayorientation; and (iii) transforming the acoustic signals from thefrequency domain to the time domain using the relative-bearing track togenerate the target track.
 4. The method of claim 3, further comprising:generating a summary report of spectral data associated with each targettrack; and outputting a compressed data report identifying thecharacteristics of the generated target track included in the generatedsummary report.
 5. The method of claim 3, wherein prior to transformingthe received acoustic signals from the time domain to the frequencydomain, the acoustic signals are conditioned for bias detection andremoval, inoperative channel shading, transient suppression, sidelobesuppression, and/or sub-aperture phase alignment.
 6. The method of claim1, wherein generating a relative-bearing beam response for each of thetransformed acoustic signals includes using linear or data adaptivemethods to compute the spatial filter weighting coefficients of thebeamformer required to generate each relative-bearing beam response. 7.The method of claim 1, wherein generating a relative-bearing beamresponse for each of the transformed signals in one or more beamsteering directions includes generating a relative bearing beam responsein one or more beam steering directions spanning the azimuthal beamspacefrom forward endfire to aft endfire.
 8. The method of claim 1, whereinthe received acoustic signals are transformed from the frequency domainto the time domain using an inverse Fast Fourier Transform algorithm. 9.The method of claim 1, wherein generating a target track furtherincludes generating a target track and scissorgram for furtherclassification.
 10. A system for autonomous jointdetection-classification, and tracking of targets of interest, thesystem comprising: (A) a microphone sensor array configured to receiveand transmit a plurality of acoustic signals; (B) a host platformincluding a memory module, a communications module, a global positioningsystem receiver and one or more embedded processors, the one or moreembedded processors configured to autonomously: (1) receive acousticsignals from a plurality of microphone sensors, each of the respectiveacoustic signals characterized by time-series data; (2) detect a targetof interest corresponding to each of the respective received acousticsignals by, (a) transforming the received acoustic signals from the timedomain to the frequency domain; (b) generating a relative-bearing beamresponse for each of the transformed signals in one or more beamsteering directions; (c) estimating the median background noise level ofthe relative-bearing beam response in each of the one or more beamsteering directions and normalizing the spectral response of therelative-bearing beam response by the estimated background noise level,wherein a normalized spectral response is produced that is associatedwith each relative-bearing beam response; (d) performing a jointdetection-classification operation, wherein the jointdetection-classification operation comprises a selective frequencyintegration of the normalized spectral response associated with eachrelative-bearing beam response, wherein the frequency integration isconstrained by the detailed physics of the underlying classifier for thetarget of interest to satisfy a spatial and spectral target hypothesisof the classifier; (e) computing a detection surface in bearing and timebased on the frequency integration of the joint detection-classificationoperation; (f) determining a decision surface in bearing and time bycalculating a constant-falsealarm-rate (CFAR) detection threshold fromthe estimated background noise level and applying theconstant-false-alarm-rate detection threshold to the detection surface;and (e) generating a target track corresponding to the decision surfaceby, (i) associating decision surface threshold exceedances to produce arelative-bearing track as a function of bearing and time; (ii)calculating a true bearing track corresponding to each of therelative-bearing tracks by reconciling the relative bearing tracks withan estimate of microphone sensor array orientation; and (iii)transforming the received acoustic signals from the frequency domain tothe time domain using the relative-bearing track; generating a summaryreport of spectral data associated with each target track; and (f)producing as an output a compressed data report identifying thecharacteristics of the generated target track included in the generatedsummary report.
 11. The system of claim 10, wherein the microphonesensor array is external to the host platform.
 12. The system of claim10, wherein the microphone sensor array is configured to be towed by thehost platform.
 13. The system of claim 10, wherein the microphone sensorarray is mounted on the host platform.
 14. The system of claim 10,wherein the host platform comprises a marine vehicle platform that ispropelled using an electric-motor driven propeller, a buoyancy-basedpropulsion system, or a propulsion system powered by wave energy. 15.The system of claim 14, wherein a separate surrogate marine vehicleplatform is further configured to transmit target of interest-likesignals from known locations allowing the quantification of parametersassociated with each generated target track including one or more ofdetection probability, detection false alarm probability, classificationprobability, classification false alarm probability, system processinggain and bearing errors.
 16. The system of claim 10, wherein themicrophone sensor array includes one of a multichannel hydrophonesensor, a linear hydrophone sensor, a planar hydrophone sensor, a mobilehydrophone sensor, a fixed hydrophone sensor, or a passive hydrophonesensor.
 17. The system of claim 10, wherein prior to transforming thereceived acoustic signals from the time domain to the frequency domain,the acoustic signals are conditioned for bias detection and removal,inoperative channel shading, transient suppression, sidelobesuppression, and/or sub-aperture phase alignment.
 18. The system ofclaim 10, wherein generating a relative-bearing beam response for eachof the transformed acoustic signals includes using linear ordata-adaptive methods to compute the spatial filter weightingcoefficients of the beamformer required to generate eachrelative-bearing beam response.
 19. The system of claim 10, whereingenerating a relative-bearing beam response for each of the transformedsignals in one or more beam steering directions includes generating arelative bearing beam response in one or more beam steering directionsspanning the azimuthal beamspace from forward endfire to aft endfire.20. The system of claim 10, wherein a time-synchronized acoustic signalis transmitted from a separate vehicle platform and used to calibratethe one or more embedded processors to achieve increased jointdetection-classification performance and decreased detection false alarmrates.